Reactive and Automatic Behavior in Plan Execution

Much of the work on execution assumes that the agent constantly senses the environment, which lets it respond immediately" to errors or unexpected events. In this paper, we argue that this purely reactive strategy is only optimal if sensing is inexpensive, and wc formulate a simple model of execution that incorporates the cost of sensing. We present, an average-case analysis of this model, which shows that in domains with high sensing cost or low probability of error, a more ’automatic’ strategy one with long inter~als between sensing can lead to less expensive execution. The analysis also shows that the distance to the goal has no effect on the optimal sensing interval. These results run counter to the prevailing wisdom in the planning community, but they promise a more balanced approach to the interleaving of execution and sensing. Reactive and Automatic Execution Much of the recent research on plan execution and control has focused on reactive systems. Onc central characteristic of such approaches is that the agent senses the environment on each time step, thus ensuring that it can react promptly to any errors or othcr unexpected events. This holds whether the agent draws on largescale knowledge structures, such as plans (Howe & Cohen, 1991)or cases (Hammond, Converse, & Marks, 1988), or bases its decisions on localized structures, such as control rules (Bresina, Drummond, & Kedar, 1993: Grefenstette, Ramsey, & Schultz, 1990) or neural networks (Sutton, 1988; Kaelbling, 1993). However, human beings still provide the best examples of robust physical agents, and the psychological literature reveals that humans do not always behave in a reactive manner. People can certainly operate in reactive or ’closed-loop’ mode, which closely couples execution with sensing (Adams, 1971). But at least in some domains, humans instead operate in automatic or ’openloop’ mode, in which execution proceeds in the absence of sensory feedback (Schmidt, 1982). Thus, at least some contexts, successful agents appear to prefer nonreactive strategies to reactive ones) One explanation for this phenomenon is that there exists a tradeoff between the cost of sensing, which models of reactivc agents t~-pically ignore, and the cost of errors that occur during execution. For some situations, the optimal sensing strategy is completely reactive behavior, in which the agent observes the environment after each execution step. For other situations, the best strategy is completely automatic behavior, in which execution occurs without sensing. In most cases, the optimum wiU presumably fall somewhere between these two extremes, with the agent sensing the world during execution, but not after every step. There exist other reasons for preferring automatic to reactive behavior in some situations. At least for humans, the former appears to require fewer attentional resources, which lets them execute multiple automatic procedures in parallel. Humans also exhibit a well-known tradeoff between speed and accuracy, and in some cases one may desire an automated, rapid response to a reactive, accurate one. However, our goal here is not to provide a detailed account of human execution strategies, but to better understand the range of such strategies and the conditions under which they are appropriate. Thus, we wiU focus on the first explanation above, which assigns an explicit cost to the sensing process. In the following pages, we attempt to formalize the tradeoff between the cost of sensing and the cost of errors, and to identify the optimal position for an agent to take along the continuum from closed-loop, reactive behavior to open-loop, automatic behavior. In the next section, we present an idealized model of execution that takes both factors into account, followed by an analysis 1 Note that the distinction between reac.ti~e and automatic behavior is entirely different from the more common distinction between reaction and deliberation. The former deals cxclusively with sensing strategies during execution, whereas the latter contrasts execution with plan generation. LANGLEY 299 From: AIPS 1994 Proceedings. Copyright © 1994, AAAI (www.aaai.org). All rights reserved. of this model. After this, we present some theoretical curves that illustrate the behavioral implications of the analysis. Finally, we discuss related work on sensing and execution, along with some prospects for future research. A Model of Execution Cost We would like some model of execution that lets us understand the tradeoff between the cost of sensing and the cost of errors. Of course, any model is necessarily an idealization of some actual situation, and from the many possible models, we must selcct one that is simultaneously plausible and analytically tractable. Thus, we will begin with a realistic scenario and introduce some simplifying assumptions that we hope retain the essential characteristics. One common problem that involves physical agents is robot navigation. In some approaches, the agent retrieves or generates a plan for moving through an environment, then executes this plan in the physical world. Unfortunately, the plan does not always operate as desired. One source of divergence from the planned path comes from actuator error: a command to turn 35 degrees or to move 5.2 meters ahead may not execute exactly as specified. Another source of divergence comes from external forces: another agent may bump into the robot or aa unexpected slope may alter its direction. Similar issues arise in the control of planes and boats, where malfunctioning effectors or unpredictable forces like wind can take the craft off the planned course. In the standard reactive control regimen, the agent senses the environment on every time step, detects errors or divergences as soon as they occur, anti takes action to put the agent back on the desired path. 2 The quality of the resulting plan execution takes into account he number of steps required to reach the goal or some similar measure. In a more general framework, execution quality also takes into account he cost c of sensing, which discourages a rational agent from unnecessary sensing and leads it to sample the environment only after every s time steps. This sensing cost m~." actually add to the execution time, or it may draw on other resources; here we care only that it somehow contributes tothe overall